9 results
LO59: Retention of critical procedural skills post-simulation training: a systematic review
- C. Legoux, R. Gerein, K. Boutis, A. Plint
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 21 / Issue S1 / May 2019
- Published online by Cambridge University Press:
- 02 May 2019, p. S29
- Print publication:
- May 2019
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Short-term gains in knowledge and skills of critical emergency procedures are demonstrated after simulation, but there is uncertainty regarding long term retention. Our objective was to determine whether simulation of critical emergency procedures promotes long term retention of procedural skills in non-surgical physicians likely to perform them. Methods: MEDLINE and Embase (from start of database to June 2018) and the CENTRAL Trials Registry of the Cochrane Collaboration (May 2018 Issue) were searched using a peer-reviewed strategy. Studies were eligible if they (1) were observational cohorts, quasi-experimental or randomized controlled trials, (2) assessed intubation, cricothyrotomy, periocardiocentesis, tube thoracostomy or central line placement performance by non-surgical physicians, (4) utilized any form of simulation (all levels of realism and technology), and (4) assessed skill performance immediately after and at ≥3 months post-simulation. There was no language restriction. Two reviewers independently assessed article eligibility. One reviewer extracted data and assessed study quality. Primary outcome was skill performance 3 months post-simulation. Secondary outcomes included skill performance at 6 and ≥12 months post-simulation, and skill competency at 3 months post-simulation. Results: 1370 citations were identified. 12 studies were eligible. Methodological quality was uniformly poor with high risk of bias, lack of defined primary outcomes, inadequate sample sizes, and non-standardized, unvalidated tools of unclear clinical significance. Given significant heterogeneity in design, populations, procedures, and outcome timing, a narrative synthesis of results was undertaken. In 10 studies participants’ performance at 3, 6 and 12 months retention testing remained above baseline assessment. However, 3 studies showed a significant decrease in performance at 3 months post-simulation compared to immediately post-simulation. Performance was also lower in 2 studies at 6 months post-simulation, and 2 studies at ≥12 months post-simulation. Four studies assessed competency and 3 demonstrated maintenance of competency. Conclusion: There was significant heterogeneity and poor methodological quality among the eligible studies. Results were conflicting for retention of procedural skills and competency. Future directions should include development of robust assessment tools, and improved research methodology of simulation education targeted at critical procedural skills.
LO86: The diagnosis of concussion in pediatric emergency departments: a prospective multicenter study
- K. Boutis, J. Gravel, S. Freedman, W. Craig, K. Tang, C. DeMatteo, S. Dubrovsky, D. Beer, G. Sangha, R. Zemek
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, pp. S37-S38
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Accurate identification of children with a concussion by emergency department (ED) physicians is important to initiate appropriate anticipatory guidance and management. In children meeting international criteria for concussion, we aimed to determine the proportion who were provided this diagnosis by the ED physician and which variables were associated with a physician-diagnosed concussion. We also compared persistent symptoms in concussion cases versus those with alternative diagnoses. Methods: This was a planned secondary analysis of a prospective, multicenter cohort study. Participants were children aged 5 through 17 years and met Zurich/Berlin International Consensus Statement criteria for concussion. The primary outcome was the proportion of study participants who were assigned a diagnosis of concussion by the treating ED physician. Based on available evidence, between 50% and 90% of children meeting international concussion criteria are also diagnosed by an ED physician as having a concussion. Assuming a worst case scenario that 50% of physicians would diagnose concussion, our anticipated study sample size of 2946 would be accompanied by a +2% margin of error at the 95% confidence level for the primary outcome. Results: Among the 2946 eligible children, 2340 [79.4% (95% CI 78.0, 80.8)] were diagnosed with a concussion by an ED physician. Twelve variables were associated with this ED diagnosis, five of which had an odds ratio (OR) > 1.5: older age (13-17 vs 5-7 years, OR=2.9), longer time to presentation (>16 vs. <16 hours, OR=2.1), nausea (OR=1.7), sport mechanism (OR=1.7), and amnesia (OR=1.6). In those with physician-diagnosed concussion versus no concussion, the frequency of persistent symptoms was 62.5% vs. 38.8% (p<0.0001) at one week, 46.3% vs. 25.8% (p<0.0001) at two weeks and 33.0% vs. 23.0% (p<0.0001) at four weeks. Conclusion: Most children meeting international criteria for concussion were provided this diagnosis by the ED physician. There were five variables which increased the odds of this diagnosis by at least 1.5-fold. Relative to international criteria, the more selective assignment of concussion by ED physicians was associated with a greater frequency of persistent concussion symptoms. Nevertheless, many patients with alternative diagnoses exhibited persistent concussive symptoms at all time points. Clinicians should therefore weigh the benefits and risks of strictly applying the Zurich/Berlin international criteria versus individual discretion.
LO41: Competency-based learning of pediatric musculoskeletal radiographs
- K. Boutis, M. Lee, M. Pusic, M. Pecarcic, B. Carrier, A. Dixon, J. Stimec
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, p. S21
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Pediatric musculoskeletal (MSK) image interpretation has been identified as a knowledge gap among emergency medicine trainees. The main objective of this study was to implement a validated on-line pediatric MSK radiograph interpretation system with a performance-based competency endpoint into pediatric emergency fellowship programs and examine the number of cases needed to achieve a competency threshold of 80% accuracy, sensitivity and specificity. We further determined proportion who successfully achieved competency in a given module and the change in accuracy from baseline to competency. Methods: This was a prospective cohort multi-centre study. There were seven MSK radiograph modules, each containing 200-400 cases (demo-https://imagesim.com/course-information/demo/). Thirty-seven pediatric emergency medicine fellows participated for 12 months. Participants did cases until they reached competency, defined as at least 80% accuracy, sensitivity and specificity. We calculated the overall and per module median number of cases required to achieve competency, proportion of participants who achieved competency, median time on case, and the mean change in accuracy from baseline to competency. Results: Overall, the median number of cases required to achieve competency was 76 (min 54, max 756). Between different body parts, there was a significant difference in the median number of cases needed to achieve competency, p <0.0001, with ankle and knee being among the most challenging modules. Proportions of those who started a module and completed it to competency varied significantly, and ranged from 32.4% in the ankle module to 97.1% in the forearm/hand, p<0.0001. The overall median time on each case was 34.1 (min 7.6, max 89.5) seconds. The overall change in accuracy from baseline to 80% competency was 13.5% (95% CI 12.1, 14.8), with the respective Cohens effect size of 1.98. The change in accuracy was different between modules, p=0.001, with post-hoc analyses demonstrating that the ankle/foot radiograph module had a greater increase in accuracy relative to elbow (p=0.009) and pelvis/femur (p=0.006). Conclusion: It was feasible for pediatric emergency medicine fellows to complete each learning pediatric MSK learning module to competency within approximately one hour, with the exception of the ankle module. Learners who completed the modules to competency demonstrated very significant increases in interpretation skill.
LO79: Climbing the learning curve teaching the pediatric emergency physician how to interpret point-of-care ultrasound images
- C. Kwan, K. Weerdenburg, M. Pecarcic, M. Pusic, M. Tessaro, H. Salehmohamed, K. Boutis
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, p. S35
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Point-of-Care Ultrasound (POCUS) is rapidly being integrated into Pediatric Emergency Medicine (PEM), and image interpretation is an important component of this skill. Currently, PEM physicians often rely on case-by-case exposure and feedback by a POCUS expert physician to learn this skill; however, this may not be efficient, reliable or feasible. Thus, there is a pressing need to develop effective POCUS image interpretation learning and assessment tools. We developed an on-line learning platform that allowed for the deliberate practice of images in four POCUS applications [soft tissue, lung, cardiac and Focused Assessment Sonography for Trauma (FAST)], and determined the quantity of participant skill acquisition by deriving performance metrics and learning curves. Methods: This was a prospective cross-sectional study administered via an on-line learning and measurement platform. Images were acquired from a pediatric emergency department and each POCUS application contained 100 still/video images. Final diagnosis of each image was determined via the consensus of three PEM POCUS experts. PEM fellow and attending study participants were recruited from the USA and Canada and were required to complete the cases of at least one application. We aimed to enroll 200 participants who had to complete a minimum of 100 cases which, based on prior work, would provide sufficient raters for item analyses and comparisons between PEM attendings and fellows. To derive reference standard performance metrics and to validate image interpretations, a unique set of five PEM POCUS experts completed each application. Results: We enrolled 225 PEM physicians, 74 fellows and 151 attendings. For all applications, the Cohens d effect size was large at 0.87, and there was no difference between PEM attendings and fellows with respect to summary performance metrics (accuracy, p= 0.29; sensitivity, p=0.13; specificity, p=0.92). Final accuracy soft tissue, lung, cardiac, and FAST for all participants was 86.4%, 89.6%, 81.6%, 88.0%, respectively, and the corresponding accuracy of PEM POCUS experts for each application was 96.0%, 96.0%, 90.0%, and 93.0%. Learning curves show maximal learning gains (inflection point) up until 65 cases for soft tissue, 70 for FAST, 75 for lung, and 85 for cardiac. Conclusion: Deliberate practice of POCUS image interpretation was effective for ensuring broad domain coverage and predictable skill improvement. Specifically, there was a large learning effect after 100 case interpretations, and 65-85 case interpretations were needed to reach an accuracy threshold of approximately 85%.
LO44: Optimizing skill retention in radiograph interpretation: a multicentre randomized control trial
- K. Boutis, B. Carrier, J. Stimec, M. Pecarcic, A. Willan, M. Pusic
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, p. S22
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Simulation-based education systems have increased physician skill in radiograph interpretation. However, the degree of skill retention and the factors that influence it are relatively unknown. The main objective of this research was to determine the rate and quantity of skill decay in post-graduate trainee physicians who completed a simulation-based learning intervention of radiograph interpretation. The impact of testing and refresher education on skill decay was also examined. Methods: This was a prospective, multicenter, analysis-blinded, four arm randomized control trial conducted from November 2014 to June 2016. Study interventions were administered using an on-line learning and measurement platform. Pediatric and emergency medicine residents in the United States and Canada were eligible for study participation. Participants were randomized to one of four groups. All participants completed an 80-case deliberately practiced learning set of pediatric elbow radiographs followed by an immediate 20-case post-test. Following this, Group 1 had no testing until 12 months; Groups 2, 3, and 4 had testing (20 cases without feedback) every 2 months until 12 months, but Group 3 also had refresher education (20 cases with feedback) at six months while Group 4 had refresher education at two, six, and ten months. The main outcome measure was accuracy at 12 months, adjusted for immediate post-test score, days to completion of 12 month test, and time on case. Based on prior data, we assumed the smallest important difference between groups in learning decay is 10%, a between-participant/within-group standard deviation of 17%, a type I error probability of 5%, a power of 80% and adjusted for three tests with a Bonferroni correction. For the primary analysis of Group 1 versus 2, 3, 4, this resulted in a minimal total sample size of 56, with 14 participants per group. Results: We enrolled 106 participants that completed all study interventions. The sample sizes in Groups 1, 2, 3, and 4 were 42, 22, 22, and 20 respectively. Overall, accuracy increased by 11.8% (95% CI 9.8, 13.8) with the 80-case learning set and then decreased by 5.5% (95% CI 2.5, 8.5) at 12 months. The difference in learning decay in Group 1 vs. Groups 2, 3, 4 was -8.1% (95% CI 2.5, 13.5), p=0.005. For Group 2 vs. Group 3 and 4, it was +0.8% (95% CI -7.2, 7.3), p=0.8, and between Group 3 vs. Group 4 it was +0.8% (95% CI -7.3, 10.1), p=0.8. Conclusion: Skill decay was significantly reduced by testing with 20 cases every two months. Refresher education had no additional effect to testing on reducing learning decay.
MP20: ImageSim - performance-based medical image interpretation learning system
- K. Boutis, M. Pecarcic, M. Pusic
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, p. S47
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Medical images (e.g. radiographs) are the most commonly ordered tests in emergency medicine. As such, emergency medicine physicians are faced with the task of learning the skill of interpreting these images to an expert performance level by the time they provide opinions that guide patient management decisions. However, discordant interpretations of these images between emergency physicians and expert counterparts (e.g. radiologists) is a common cause of medical error. In pediatrics, this problem is even greater due to the changing physiology with age. Methods: ImageSim (https://imagesim.com/) is an evidence-based on-line learning platform derived and validated over an 11 year period (https://imagesim.com/research-and-efficacy/). This learning system incorporates the concepts of cognitive simulation, gamification, deliberate practice, and performance-based competency in the presentation and interpretation of medical images. Specifically, ImageSim presents images as they are experienced in clinical practice and incorporates a normal to abnormal ratio is representative of that seen in emergency medicine. Further, it forces the participant to commit to the case being normal or abnormal and if abnormal, the participant has to visually locate the specific area of pathology on the image. The participant submits a response and gets text and visual feedback with every case. After each case, the participant gets to play again until they reach a desired competency threshold (80% is bronze resident; 90% silver staff emergency medicine physician; 97% gold radiologist). Importantly, the learning experience also emphasizes deliberate practice such that the learning system provides hundreds of case examples and therefore each participants performance has the opportunity to improve along their individual learning curve. Results: Course selection was made based on known medical image interpretation knowledge gaps for practicing emergency physicians. Currently, ImageSim live courses include pediatric musculoskeletal radiographs (2,100 cases, 7 modules) and pediatric chest radiographs (434 cases). In 2018, we will also release a pediatric point-of-care ultrasound course (400 cases, 4 modules) and the pre-pubertal female genital examination (150 cases). For a demo, go to https://imagesim.com/demo/. Using ImageSim, the deliberate practice of about 120 cases (1 hour time commitment) increases accuracy on average by 15%. Currently integrated into 10 emergency medicine training programs and there are about 300 continuing medical education world-wide participants. Conclusion: While acquiring mastery for these images may take years to acquire via clinical practice alone, this learning system can potentially help achieve this in just a few hours.
LO81: Bridging the GAP: A deliberate practice method for learning Genital Abnormalities in Prepubescent girls
- K. Boutis, A. Davis, M. Pecarcic, M. Pusic, M. Shouldice, T. Smith, J. Brown
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 20 / Issue S1 / May 2018
- Published online by Cambridge University Press:
- 11 May 2018, pp. S35-S36
- Print publication:
- May 2018
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Correctly identifying pathology in pre-pubertal females is a high-stakes physical examination skill. Currently, learning this skill relies heavily on case-by-case exposure, which is variable, limited and often results in suboptimal skill. Thus, there is a need to develop and evaluate learning platforms that simulate the presentation and diagnosis of this important clinical task. We developed an on-line learning and assessment platform that allowed the deliberate practice of 158 pre-pubertal female genital image interpretations . We examined the quantity of skill acquisition by deriving performance metrics and learning curves. Methods: This was a prospective cross-sectional study administered via an on-line learning and assessment platform. Colposcopic images were acquired from a child abuse clinic. Two child abuse experts interpreted images to determine case solutions and 40% of cases had medical or traumatic pathology. Further, to validate image interpretations, a unique set of five child abuse and pediatric gynaecology experts reviewed the cases. Study participants were recruited from the USA and Canada and were required to complete all 158 cases. For each image, learners designated cases as normal or abnormal and if abnormal indicated the abnormal area on the image. The primary outcome was the change in accuracy, sensitivity and specificity. Results: We enrolled 107 participants, 26 medical students, 31 pediatric residents, 24 pediatric emergency fellows, and 26 pediatric emergency attendings. For all participants, the change in accuracy was +9.6% for accuracy (<0.001), +1.4% for sensitivity (p=0.6) and +15.7% (p<0.001) for specificity. The final score for accuracy, sensitivity and specificity was 79.5%, 66.1%, and 87.8%, respectively. There was no difference between learner types with respect to summary performance metrics (accuracy, p=0.15; sensitivity, p=0.44; specificity, p=0.54). Learning curves show maximal learning gains (inflection point) up until 100 cases. Conclusion: Deliberate practice of pre-pubertal female image interpretation was effective for ensuring predictable skill improvement for normal cases but was less effective for abnormal cases. Future research could examine how to refine the education tool to better serve diagnostic skill of abnormal cases.
P016: Low risk ankle rule, high reward-a quality improvement initiative to reduce ankle x-rays in the pediatric emergency department
- F. Al-Sani, M. Ben-Yakov, G. Harvey, J. Gantz, D. Jacobson, K. Boutis, O. Ostrow, T. Principi
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 19 / Issue S1 / May 2017
- Published online by Cambridge University Press:
- 15 May 2017, p. S83
- Print publication:
- May 2017
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Our tertiary care institution embarked on the Choosing Wisely campaign to reduce unnecessary testing, and selected the reduction of ankle x-rays as part of its top five priority initiatives. The Low Risk Ankle Rule (LRAR), an evidence-based decision rule, has been derived and validated to clinically evaluate ankle injuries which do not require radiography. The LRAR, is cost-effective, has 100% sensitivity for clinically important ankle injuries and reduces ankle imaging rates by 30-60% in both academic and community setting. Our objective was to significantly reduce the proportion of ankle x-rays ordered for acute ankle injuries presenting to our pediatric Emergency Department (ED). Methods: Medical records were reviewed for all patients presenting to our tertiary care pediatric ED (ages 3- 18 years) with an isolated acute ankle injury from Jan 1, 2016-Sept 30, 2016. Children with outside imaging, an injury that occurred >72 hours prior, or those who had a repeat ED visit for same injury were excluded. Quality improvement (QI) initiatives included multidisciplinary staff education about the LRAR, posters placed within the ED highlighting the LRAR, development of a new diagnostic imaging requisition for ankle x-rays requiring use of the LRAR and collaboration with the Division of Radiology to ensure compliance with new requisition. The proportion of patients presenting to the ED with acute ankle injuries who received x-rays was measured. ED length of stay (LOS), return visits to the ED and orthopedic referrals were collected as balancing measures. Results: At baseline 88% of patients with acute ankle injuries received x-rays. Following our multiple interventions, the proportion of x-rays decreased significantly to 54%, (p<0.001). This decrease in x-ray rate was not associated with an increase in ED LOS, ED return visits or orthopedic referrals. There was an increase uptake of the dedicated x-ray requisition over time to 71%. Conclusion: This QI initiative to increase uptake of the LRAR, resulted in a significant reduction of ankle x-rays rates for children presenting with acute ankle injuries in our pediatric ED without increasing LOS, return visits or need for orthopedic referrals for missed injuries. Just as in the derivation and validation studies, the reductions have been sustained and reduced unnecessary testing and ionizing radiation.
LO25: How safe are our pediatric emergency departments? A multicentre, prospective cohort study
- A. Plint, L. Calder, Z. Cantor, M. Aglipay, A.S. Stang, A.S. Newton, S. Gouin, K. Boutis, G. Joubert, Q. Doan, A. Dixon, R. Porter, S. Sawyer, M. Bhatt, K. Farion, T. Crawford, D. Dalgleish, D.W. Johnson, T. Klassen, N. Barrowman, for Pediatric Emergency Research Canada
-
- Journal:
- Canadian Journal of Emergency Medicine / Volume 19 / Issue S1 / May 2017
- Published online by Cambridge University Press:
- 15 May 2017, p. S36
- Print publication:
- May 2017
-
- Article
-
- You have access Access
- Export citation
-
Introduction: Data regarding adverse events (AEs) (unintended harm to the patient from health care provided) among children seen in the emergency department (ED) are scarce despite the high risk setting and population. The objective of our study was to estimate the risk and type of AEs, and their preventability and severity, among children treated in pediatric EDs. Methods: Our prospective cohort study enrolled children <18 years of age presenting for care during 21 randomized 8 hr-shifts at 9 pediatric EDs from Nov 2014 to October 2015. Exclusion criteria included unavailability for follow-up or insurmountable language barrier. RAs collected demographic, medical history, ED course, and systems level data. At day 7, 14, and 21 a RA administered a structured telephone interview to all patients to identify flagged outcomes (e.g. repeat ED visits, worsening/new symptoms, etc). A validated trigger tool was used to screen admitted patients’ health records. For any patients with a flagged outcome or trigger, 3 ED physicians independently determined if an AE occurred. Primary outcome was the proportion of patients with an AE related to ED care within 3 weeks of their ED visit. Results: We enrolled 6377 (72.0%) of 8855 eligible patients; 545 (8.5%) were lost to follow-up. Median age was 4.4 years (range 3 months to 17.9 yrs). Eight hundred and seventy seven (13.8%) were triaged as CTAS 1 or 2, 2638 (41.4%) as CTAS 3, and 2839 (44.7%) as CTAS 4 or 5. Top entrance complaints were fever (11.2%) and cough (8.8%). Flagged outcomes/triggers were identified for 2047 (32.1%) patients. While 252 (4.0%) patients suffered at least one AE within 3 weeks of ED visit, 163 (2.6%) suffered an AE related to ED care. In total, patients suffered 286 AEs, most (67.9%) being preventable. The most common AE types were management issues (32.5%) and procedural complications (21.9%). The need for a medical intervention (33.9%) and another ED visit (33.9%) were the most frequent clinical consequences. In univariate analysis, older age, chronic conditions, hospital admission, initial location in high acuity area of the ED, having >1 ED MD or a consultant involved in care, (all p<0.001) and longer length of stay (p<0.01) were associated with AEs. Conclusion: While our multicentre study found a lower risk of AEs among pediatric ED patients than reported among pediatric inpatients and adult ED patients, a high proportion of these AEs were preventable.